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ABSTRACT 



Techniques for determining an output rate for a bit stream, 
the output rate being determined by applying information 
read from the bit stream to a model of a receiver for the bit 
stream. The techniques are employed to construct a statis- 
tical multiplexer that multiplexes varying bit-rate MPEG-2 
bit streams onto a satellite up-link. Minimum and maximum 
output rates for each MPEG-2 bit stream are determined 
such that neither a queue for the bit stream in the multiplexer 
nor the bit stream's decoder will underflow or overflow. The 
rates with regard to the decoder are determined using liming 
information read from the bit stream and a general model of 
an MPEG-2 decoder. The multiplexer first allocates each bit 
stream its minimum bandwidth and then allocates any 
remaining bandwidth to the bit streams in proportion to the 
diflference between the minimum and maximum output rates 
for the bit streams, with no bit stream receiving more than 
its maximum output rate. If there is not enough bandwidth 
to give every bit stream its minimum rate, bit rates are 
allocated according to priorities assigned to the bit streams. 

23 Claims, 10 Drawing Sheets 
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USING A RECEIVER MODEL TO 
MULTIPLEX VARIABLE-RATE BIT 
STREAMS HAVING TIMING CONSTRAINTS 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The invention has to do with the transmission of variable- 
rate bit streams, generally and more particularly with the 
efEcient time multiplexing of several such bit streams onto 
a transmission medium. 

2. Description of the Prior Art: FIGS. 1-3 

Anew problem in data transmission is the transmission of 
data that requires a high band width, is bursty, and has 
temporal constraints. Traditionally, data transmission has 
been done on the public switched networks provided by the 
telephone companies and on packet networks. The pub he 
switched networks are designed for interactive voice 
applications, and so provide relatively low-bandwidth cir- 
cuits that satisfy stringent temporal constraints. The packet 
networks are designed for the transfer of data between 
computer systems. The only constraint is that the data 
eventually arrive at its destination. The amount of bandwidth 
available for a transfer depends on the degree of congestion 
in the network. The packet networks thus typically make no 
guarantees whatever about when or even in what order the 
data in a burst of data will arrive at its destination. As may 
be seen from the foregoing, neither the telephone network 
nor the packet network is well-adapted to handle high- 
bandwidth bursty data with time constraints. An example of 
such data is digital television which has been compressed 
according to the MPEG-2 standard. For details on the 
standard, see Background Information on MPEG-land 
MPEG-lTelevision Compression, which could be found in 
November 1996 at the URL http://w^vw.cdrevo lution.com/ 
texl/mpeginfo .htm . 

FIG. 1 shows those details of the MPEG-2 standard that 
are required for the present discussion. The standard defines 
a encoding scheme for compressing digital representations 
of video. The encoding scheme takes advantage of the fact 
that video images generally have large amounts of spatial 
and temporal redundancy. There is spatial redundancy 
because a given video picture has areas where the entire area 
has the same appearance; the larger the areas and the more 
of them there are, the greater amount of spatial redundancy 
in the image. There is temporal redundancy because there is 
often not much change between a given video image and the 
ones that precede and follow it in a sequence. The less the 
amount of change between two video images, the greater ihe 
amount of temporal redundancy. The more spatial redun- 
dancy there is in an image and the more temporal redun- 
dancy there is in the sequence of images to which the image 
belongs, the fewer the bits that will be needed to represent 
the image. 

Maximum advantage for the transmission of images 
encoded using the MPEG-2 standard is obtained if the 
images can be transmitted at variable bit rates. The bit rates 
can vary l>ecause the rate at which a receiving device 
receives images is constant, while the images have varying 
number of bits. A large image therefore requires a higher bit 
rate than a small image, and a sequence of MPEG images 
transmitted at variable bit rates is a variable -rate bit stream 
with time constraints. For example, a sequence of images 
that shows a "talking head" will have much more spatial and 
temporal redundancy than a sequence of images for a 
commercial or MTV song presentation, and the bit rate for 
the images showing the "talking head" will be far lower than 
the bit rate for the images of the MTV song presenUtion. 



2,384 

2 

The MPEG-2 compression scheme represents a sequence 
of video images as a sequence of pictures, each of which 
must be decoded at a specific time. There are three ways in 
which pictures may be compressed. One way is intra-coding, 

5 in which the compression is done without reference to any 
other picture. This encoding technique reduces spatial 
redundancy but not time redundancy, and the pictures result- 
ing from it are generally larger than those in which the 
encoding reduces both spatial redundancy and temporal 

jQ redundancy. Pictures encoded in this way are called 
I-pictures. A certain number of I-pictures are required in a 
sequence, first, because the initial picture of a sequence is 
necessarily an I-piclure, and second, because I-pictures 
permit recovery from transmission errors. 

15 Time redundancy is reduced by encoding pictures as a set 
of changes from earlier or later pictures or both. In MPEG-2, 
this is done using motion compensated forward and back- 
ward predictions. When a picture uses only forward motion 
compensated prediction, it is called a Predictive -coded 

9Q picture, or P picture. When a picture uses both forward and 
backward motion compensated predictions, it is called a 
Bidirectional predictive-coded picture, or a B picture in 
short. P pictures generally have fewer bits than I pictures and 
B pictures have the smallest number of bits. The number of 

95 bits required to encode a given sequence of pictures in 
MPEG-2 is thus dependent on the distribution of picture 
coding types mentioned above, as well as the picture content 
itself. As will be apparent from the foregoing discussion, the 
sequence of pictures required to encode the images of the 

30 "talking heads" will have fewer and smaller I pictures and 
smaller B and P pictures than the sequence required for the 
MTV song presentation, and consequently, the MPEG-2 
representation of the images of the talking heads will be 
much smaller than the MPEG-2 representation of the images 

35 of the MTV sequence. 

The MPEG-2 pictures are being received by a low-cost 
consumer electronics device such as a digital television set 
or a set-top box provided by a CATV service provider. The 
low cost of the device strictly limits the amount of memory 

40 available to store the MPEG-2 pictures. Moreover, the 
pictures are being used to produce moving images. The 
MPEG-2 pictures must consequently arrive in the receiver in 
the right order and with time intervals between them such 
that the next MPEG-2 picture is available when needed and 

45 there is room in the memory for the picture which is 
currently being sent. In the art, a memory which has run out 
of data is said to have underflowed, while a memory which 
has received more data than it can hold is said to have 
overflowed. In the case of underflow, the motion in the TV 

50 picture must stop until the next MPEG-2 picture arrives, and 
in the case of overflow, the data which did not fit into 
memory is simply lost. 

FIG. 1 is a representation of a digital picture source 103 
and a television 117 that are connected by a channel 114 that 

55 is carrying a MPEG-2 bit stream representation of a 
sequence of TV images. In system 101, a digital picture 
source 103 generates uncompressed digital representations 
of images 105, which go to variable bit rate encoder 107. 
Encoder 107 encodes the uncompressed digital representa- 

60 tions to produce variable rate bit stream 109. Variable rate bit 
stream 109 is a sequence of compressed digital pictures 111 
of variable length. As indicated above, when the encoding is 
done according to the MPEG-2 standard, the length of a 
picture depends on the complexity of the image it represents 

65 and whether it is an I picture, a P picture, or a B picture. 
Additionally, the length of the picture depends on the 
encoding rate of VBR encoder 107. That rate can be varied. 
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Id general, ihe more bits used to encode a picture, the better channel 114 that have varying bit rates and time constrainis, 

the picture quality. slot 213 for each of the channels 114 must output enough 

Bit stream 109 is transferred via a channel 114 to VBR Packets to provide bits at the rate ncce^ary to send the 

decoder 115. which decodes the compressed digital pictures '"8«^«' P''^"'^^ 1" '° '^^l"'"'' within channel 114 s 
111 to produce uncompressed digi.al pictures 105. These in 5 '^F^.^- °^'=ffl°^- 1"** ^derflow constmints. Of cour^ mos 

^ -J J * * 1 • * Tf * 1 • • • of the time, a channel s slot 213 will be outputtmg fewer 

turn are provided to television 117^ If television 117 is a ^^^^^^ ^^^^ maximum to medium 207, and sometimes 

digital television, they will be provided directly; othenvise ^^^^ j^^j^ ^^^1^ ^j^^ 213 

there will be another element which converts uncompressed "esents a fixed portion of medium 207's total bandwidth, 

digital pictures 105 mto standard analog television signals 3 ^i^j 213 is not fuU, a part of medium 207*s 
and then provides those signals to television 117. There may 10 bandwidth is being wasted. 

of course be any number of decoders 115 receiving the ^^^^^ ^^^-^ ^^^^ bandwidth of medium 207, 

output of a single encoder 107. ^ technique is used which ensures that time slice 211 is 

la FIG. 1, channel 114 transfers bit stream 109 as a generally almost full of packets. This technique is termed 

sequence of packets 113. The compressed digital pictures statistical multiplexing. It takes advantage of the fact that at 

111 thus appear in FIG. 1 as varying-length sequences of ^ ^^^^^ moment of time, each of the channels in a set of 

packets 113. Thus, picture 111(£0 has n packets while picture channels will be carrying bits at a different bit rate, and the 

111(0) has k packets. Included in each picture Ul is timing bandwidth of medium 207 need only be large enough at that ■ 

information 112. Timing information 112 contains two kinds moment of time to transmit what the channels are presently 

of information: clock information and time stamps. Clock carrying, not large enough to transmit what all of the 

information is used to synchronize decoder 115 with encoder channels could carry if they were transmitting at the maxi- 

107. The time stamps specify when a picture is to be ^ate. The output of the channels is analyzed statisti- 

decoded and when it is actually to be displayed. The times ^ally to determine what the actual maximum rate of output 

specified in the Ume stamps are specified in tenms of the ^^e entire set of channels will be and the bandwidth of 

clock information. As indicated above, VBR decoder 115 medium 207 is sized to satisfy that actual peak rate, 

contains a relatively small amount of memory for storing Typically, the bandwidth that is determined in this fashion 

pictures 113 untU they are decoded and provided to TV 117. ^e far less than is required for multiplexing in the 

This memory is shown at 119 in FIG. 1 and is termed in the manner shown at 210 in FIG. 2. As a result, more channels 

following the decoder's bit buffer. Bit buffer 119 must be at ^an be sent in a given amount of bandwidth. At the level of 

least large enough to hold the largest possible MPEG-2 ^^^^^ ^^at statistical multiplexing requires is a mechanism 
picture. Further, channel U4 must provide the pictures 111 ' ^^ich in effect permits a channel 114 to have a slot in lime 

to bit buffer 119 in such fashion that decoder 115 can make ^^q^ 2II which varies in length to suit the acmal needs of 

them available at the proper times to TV 117 and that bit channel 114 during that time slice 2U. Such a time slice 211 

buffer 119 never overflows or underflows. Bit buffer 119 ^j^h varying-length slots 215 is shown at 214. 

underflows if not all of the bits in a picture 111 have arrived ^^j^od of statistically multiplexing bit streams is 

in bit buffer 119 by the time specified m the picture's time disclosed in Rao, U.S. Pat. No. 5,506,844. Method for 

stamp for decoder 115 to begm decoding the picture Ul. Configuring a Statistical Multiplexer to Dynamically Alio- 

Providing pictures 111 to VBR decoder 115 in the proper cate Communication Channel Bandwidth, issued Apr. 9. 

order and at the proper times is made more complicated by 1995. piG. 3 is an overview of the method disclosed in the 

the fact that a number of channels 114 may share a single Rao application. System 301 includes a set of encoders 

very high bandwidth data link. For example, a CATV 302(0 which encode a set of bit streams 105(0 ... /i). 

provider may use a satellite link to provide a large number During a given period of time, termed herein a window, each 

of TV programs from a central location to a number of encoder 302(0 encodes at a constant bit rate; however, the 

CATV network head ends, from which they are transmitted bit rate may be changed at the beginning of the window. The 

via coaxial or fiber optic cable to individual subscribers or output of an encoder 302(0 is thus a bit stream 108(0 having 

may even use the satellite link to provide the TV programs a piccewise-constant bit rate. The bit streams 108(0 . . . /i) 

directly to the subscribers. When a number of channels share are input to multiplexer 303, which multiplexes them onto 

a medium such as a satellite link, the medium is said to be medium 207. 

multiplexed among the channels. Multiplexer 303 maximizes the use of medium 207 by 
FIG. 2 shows such a multiplexed medium. A number of so adjusting the bit rates of encoders 302(0 ... /a). As 

channels 114(0) through 114(rt) which are carrying packets mentioned above, there is a relationship between bit rate and 

containing bits from variable rate bit stream. 109(0 . . .n) are picture quality. Generally, the higher the bit rate, the better 

received in multiplexer 203. which processes the packets as the picture quality. Consequently, in adjusting the bit rates of 

required to multiplex them onto high bandwidth medium encoders 302(0 ... /i), multiplexer 303 must be aware of the 
207. The packets then go via medium 207 to demultiplexer 55 current picture quality of each bit stream and must adjust the 

209, which separates the packets into the packet streams for bit rates not only to maximize the use of medium 207, but 

the individual channels 114(0 .. ./j). A simple way of sharing also to maximize the picture quality of each of the bit 

a high bandwidth medium among a number of channels that streams 108(0- 

are carrying digital data is to repeatedly give each individual As Mux 303 operates, it receives information from each 
channel 114 access to the high bandwidth medium for a short 50 encoder 107(0 that indicates the picture distortion rate for 

period of time, termed herein a slot. encoder 107(0*s current encoding rate (DIF 311(0) and also 

One way of doing this is shown at 210 in FIG. 2. The short keeps track of the fullness of encoding buffer 307(0 in 

period of time appears at 210 as a slot 213; during a slot 213, encoder 107(0, as shown by arrow EBF 309(0- Encoding 

a fixed number of packets 113 belonging to a channel 114 buffer 307(0 holds bit stream 105(0 while it is being 
may be output to medium 207. Each channel 114 in turn has 65 encoded, and encoder 107(0 ro^i^l encode at a rate such that 

a slot 213, and all of the slots taken together make up a time encoding, buffer 307(0 neither overflows nor underflows, 

slice 211. When medium 207 is carrying channels like Multiplexer 303 detennines from the current distortion rates 
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of ihe encoders 107 which encoders need lo encode ai a In another aspect of the invention, the multiplexer further 

higher bit rale and which can encode at a lower bit rate and uses priority information about the bit streams in adjusting 

at the beginning of a window, it adjusts the rate of each the rates of the bit streams when there is not enough 

encoder 107, as indicated by the arrows BRCTL 305(0 . . . bandwidth for all of the bit streams to receive the minimum 

rt). to maximize the picture quality for all of the encoders 5 amount. Each bit stream is either a priority one, priority two, 

107 while maximizing the degree to which medium 207' s or priority three bit stream, with priority one bit streams 

bandwidth is used. When multiplexer 303 reduces or being required to meet the time constraints set in the bit 

increases an encoder 107(0 's bit rate, it also reduces or stream, priority two bit streams being required only not to 

increases the size of EBUF 307(/) in the encoder. underflow, and priority three bit streams being permitted to 

While the statistical multiplexer of Rao does maximize 10 underflow. Where bandwidth is necessary for priority one or 

the degree to which medium 207's bandwidth is used, it has priority two bit streams, it is taken from priority two and/or 

a number of shortcomings. Perhaps the most important of priority three bit streams. 

these is that it adjusts the multiplexing by changing picture still another aspect of the invention is a multiplexer for 

quality. The system thus cannot guarantee any user a given MPEG-2 bit streams. The statistical multiplexer multiplexes 

quality of picture. is a plurality of MPEG-2 bit streams and constant bit rate audio 

Another shortcoming is that it requires encoders that bit streams onto an output medium. Included in this aspect 

encode digital images as piecewise-constant bit streams. of the invention are a receiver model for MPEG-2 bit 

Such bit streams have a lower degree of compression than streams and algorithms for using the model to determine the 

variable-rate bit streams; further, the encoding rale and output bit rate for a MPEG-2 bit stream, 

therefore the quality of the picture changes at the beginning 20 j^^^ ^^^j. aspects and objects of the invention v^U 

of each window; with sequences of fast changing images, become apparent to those skilled in the arts lo which the 

this will produce coding artifacts in the pictures. invention pertains upon perusal of the foUowing Detailed 

Still another is that the multiplexing requires feedback Description and Drawing, wherein: 
from multiplexer 303 to encoders 302(0 . . . n). One 

consequence of this fact is that multiplexer 303 will not 25 BRIEF DESCRIPTION OF THE DRAWING 
work with pre-stored sequences of pictures 111; another is , • , , • • 
that in order to use information like encoder buffer fullness FIG- 1 ^ diagram showing how digital television 
309 and distortion information 311 to allocate bandwidth in pictures are encoded, transmitted, and decoded; 
medium 207, multiplexer 303 must take into account the FIG. 2 is a diagram showing multiplexing of variable -rate 
inner workings of encoder 107. A third is that there must be 30 bit streams onto a high band width medium; 
a high-speed connection between multiplexer 303 and each pjc. 3 is a block diagram of a prior-art statistical multi- 
encoder 302 to exchange the control information. Finally, plexer; 

the bitrate switching of the encoders and the multiplexer is ^ ^ ^ ^lock diagram of a statistical multiplexer 

difiBcuU to implement, particularly if it is necessary to which implements a preferred embodiment of the invention; 

support video inputs having different frame rates. 35 ^ ^ ^ ^^^^ ^^^^^^^ ^^^^^ ^.^^^^^ ^ 

It is an object of the invention disclosed herem to over- ^^^^^.^^^ multiplexer of FIG. 4; 

come these shortcomings and thereby to provide an ^ r u ^ a* ^^t^™«^ 

ved statistical multi lexer FIG. 6 is pseudo-code for the algorithm used to determine 

^ F ■ ^^^g ^ channel in the preferred embodiment; 

SUMMARY OF THE INVENTION FIG. 7 is a flow chart for the algorithm used to allocate the 

The invention is embodied in a multiplexer that over- total bit rate of medium 207 among the channels; 

comes the foregoing problems of the prior an by determin- g ^ conceptual block diagram of the statistical 

ing the rate at which each bit stream is to be output to the mu hip lexer; 

medium in accordance with rate information obtained by ^ .'^ ^ high-level block diagram of an encoding 

applying a model of a receiver for each of the bu streams to 45 ^^.^^ .^^^^^ implementation of the statistical 

information contained m the bit stream itself. The rate for - 

each bit stream is determined such that the receiver for the „ . ^ - r u - 1 . r 

bit stream will neither underflow nor overflow and the FIG. 10 is a more detailed view of the miplementation of 

multiplexer's use of the medium's bandwidth is maximized. statistical multiplexer; 

Advantages of the mulupiexer of the invention include the 50 FIG. 11 is a detailed view of a channel input block in the 

fact that all of the information needed to determine the rates statistical multiplexer of FIG. 10; and 

is available within the multiplexer itself, the fact that the piG 12 is a flowchart of the minimal bitrate algorithm, 

technique will work with any kind of bit stream for which it reference numbers in the drawings have at least three 

is possible to make a receiver model that is able to determine ^.^^^ rightmost digits are reference numbers 

an output rate on the basis of information read from the bit 55 ^-^^^^ g g ^^^^ to the left of those digits are the . 

stream, and the fact that adjustment of the output rates does ^^^^^ ^^^^ ^^.^^ -^^^ identified by the 

not affect the contents of the bit stream. reference number first appears. For example, an item with 

In another aspect, the rate information for each bit stream reference number 203 first appears in FIG. 2. 
is provided as a minimum allowable rate and a maximum 

allowable rate and the multiplexer allocates bandwidth to the 60 DETAILED DESCRIPTION 
bit streams by first giving each bit stream its minimum 

allowable rate and then allocating any remaining bandwidth The following Detailed Description will first present an 

to the bit streams up to the maximum rate for each bit overview of the preferred embodiment, wUI then provide a 

stream. In the preferred embodiment, allocation of the description of the hardware in which the preferred embodi- 
remaining bandwith for a bit stream is done proportionaUy 65 ment is implemented, and will finally provide a detaUed 

to the difference between the maximum and minimum rates description of the algorithms used to allocate bandwidth in 

for the bit stream. the preferred embodiment. 
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Conceptual Overview: FIG, 8 

RG. 8 presents a conceptual overview of a statistical 
multiplexer 801 which incorporates the principles of the 
invention. A number n of variable-rate bit streams 109 are 
received in receiver 803, which provides them to bandwidth 5 
portion controller 805. Bandwidth portion controller 805 
dynamically determines what portion of the bandwidth of 
medium 801 that each bit stream 109(i) is to receive and 
provides a corresponding portion 815(i) of the bit stream to 
transmitter 817, which outputs the portions 815(0 ... /i) it 
receives of each bit stream 109(0 . . . «) onto medium 207. 

Bandwidth portion controller 805 has a number of sub- 
components. There is a transmission controller 807(f) for 
each bit stream 109(i). Each transmission controller 807(i) 
contains a bit stream analyzer 809(0 and a receiver model 
811(t)- Bit stream analyzer 809(z) collects information from 
bit stream 109(0 applies receiver model 811(0 to the 
collected information to determine what rate is required by 
the condition of the receiving device. In the case of a 
MPEG-2 bit stream, the receiving device is a decoder 115(0, 
and for such a decoder, the required rate can be determined 
from the lime stamps and the sizes of the pictures making up 
bit stream 109(0- Transmission controller 807(0 applies 
receiver model 811(0 information to determine rate 

information 812(0- Bandwidth allocator 813 receives rate 
information 812(0 , . . n) and uses this information to 
allocate the portion of the bandwidth of medium 207 that 
each bit stream 109(0 is to receive. Having done this for 
each bit stream 109(0 . . .n), it provides a bit stream portion 
815(0 lhat corresponds to the allocated bandwidth to trans- 
mitter 817. 

It is worth noting here that all of the information required 
by the above technique for allocating bandwidth can be 
obtained by applying the receiver models 811 to the infor- 
mation received from the bit streams 109 and that informa- 
tion need only be exchanged between bandwidth allocator 
813 and transmission controllers 807. There is no need 
whatever to receive information from or provide information 
to the encoders 107. Put another way, all of the information 
needed to allocate the bandwidth is available within statis- 
tical multiplexer 801 itself. 

It is also worth noting that the technique of using a model 
of a receiver to control the rate at which a bit stream is output 
to a receiver may be apphed in other situations. For example, 
a receiver model could be used to control the rate at which 
a MPEG-2 encoder encoded data. 
Overview of a Preferred Embodiment: FIG. 4 

HG. 4 provides an overview of a statistical multiplexer 
401 for MPEG-2 bit streams which is implemented accord- 
ing to the principles of the invention. The main components 
of multiplexer 401 are packet collection controller 403, a 50 
transmission controller 407(0 for each variable-rate bit 
stream 109(0, a packet dehvery controller 419, and a 
modulator 423, which receives the output of packet delivery 
controller 419 and outputs it in the proper form for trans- 
mission medium 207, Packet collection controller 403 col- 55 
lects packets from variable-rate bit streams 109(0 . . , /i) and 
distributes the packets that carry a given bit stream 109(0 to 
the bit stream's corresponding transmission controller 407 
(0- In the preferred embodiment, the packets for all of the bit 
streams 109(0 . . . n) are output to bus 402. Each packet 60 
contains an indication of which bit stream it belongs to, and 
packet collection controller responds to the indication con- 
tained in a packet by routing it to the proper transmission 
controller 407(0- should be noted here that the packets in 
each bit stream 109(0 arrive in transmission conU-oUer 65 
407(0 Ih® order in which they were sent by encoder 
107(0- 
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Transmission controller 407(0 determines the rate at 
which packets from its corresponding bit stream 109(0 
output to mediimn 207. The actual rate determination is made 
by transmission rate controller 413, which at a minimum, 
bases its determination on the following inforthation: 
for at least a current picture 111 in bit stream 109(0, 
timing information 112 and the size of the current 
picture. 

a Video Buffer Verifier (VBV) model 415(0, which is a 
model of a hypothetical bit buffer 11 9(/). 
VBV model 415(0 "^es the timing information and picture 
size information to determine a range of rates at which bit 
stream 109(0 ™st be provided to the decoder's bit buffer 
119(0 if bit buffer 119(0 is to neither overflow nor under- 
flow. Transmission rate controller 413(0 provides the rate 
information to packet delivery controller 419, which uses 
the information from all of the transmission controllers 407 
to determine during each time slice how the bandwidth of 
transmission medium 207 should be allocated among the bit 
20 streams 109 during the next time slice. The more packets a 
bit stream 109(/) needs to output during a time shce, the 
more bandwidth it receives for that time slice. 

Continuing in more detail, transmission controller 407 
obtains the timing and picture size information by means of 
25 bit stream analyzer 409, which reads bit stream 109(0 as it 
enters transmission controller 407 and recovers the timing 
information 114 and the picture size 411 from bit stream 
109(0. Bit stream analyzer 409 can do so because the 
MPEG-2 standard requires that the beginning of each picture 
30 111 be marked and that the timing information 114 occupy 
predetermined locations in each picture 111. As previously 
explained, timing information 114 for each picture 111 
includes a clock value and a decoding . time stamp. Trans- 
mission controller 407(0 later decoder 115(0 ^se the 
35 clock value to synchronize themselves with encoder 107(0- 
The liming information is found in the header of the PES 
packet that encapsulates the compressed video data. The 
information is contained in the PTS and DTS time stamp 
parameters of the PES header. The MPEG-2 standard 
40 requires that a time stamp be sent at least every 700 msec. 
If a compressed picture is not expUcilly sent with a com- 
pressed picture, then the decoding time can be determined 
from parameters in the Sequence and Picture headers. For 
details, see Annex C of ISO/IEC 13818-1. Bit stream 
45 analyzer 409 determines the size of a picture simply by 
counting the bits (or packets) from the beginning of one 
picture to the beginning of the next picture. 

The timing information and the picture size are used in 
VBV model 415(0- VBV model 415(0 requires the timing 
information and picture size information for each picture in 
bit stream 109(0 from the time the picture enters multiplexer 
401 until the time the picture is decoded in decoder 115(0- 
DTS buffer 414 must be large enough to sold the timing 
information for all of the pictures required for the model. It 
should be noted here that VBV model 415(0*5 behavior is 
defined solely by the semantics of the MPEG-2 standard, not 
by any concrete bit buffer 119(0- Any bit buffer for a 
working MPEG-2 decoder must be able to provide the 
decoder with the complete next picture at the time indicated 
by the picture's timing information; that means that the bit 
buffer 119(0 for any working MPEG-2 decoder must be at 
a minimum large enough for the largest possible MPEG-2 
picmre. Given this minimum buffer size, the timing infor- 
mation for the pictures, and the sizes of the individual 
pictures, VBV model 415(0 cao determine a rate of output 
for bit stream 109(0 which will guarantee for bit buffers 
119(0 of any working MPEG-2 decoder that each picture 
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arrives in the bit buffer 119(/) before the time it is to be 
decoded and that there will be no overflow of bit buffer 
119(0. 

Details of Transmission Controller 407 and Packet Delivery 
ConlroUer 419: FIG. 5 

FIG, 5 shows the details of a preferred embodiment of 
transmission controller 407 and packet delivery controller 
419. The figure shows three of the n transmission 
controllers, namely transmission controllers 407(. . . k), and 
the two major components of packet delivery controller 419, 
namely central bit rate controller 501 and switch 511. 
Beginning with transmission controller 407(/), in addition to 
transmission rate controller 413, analyzer 409, and VBV 
model 415, transmission controller 409 includes statistical 
multiplexer buffer (SMB) 507, a meter 505 for buffer 507, 
and throttle 509. 

SMB 507(0 is a first -in-first-oul pipe buffer which holds 
the bits of bit stream 109(i) while ihey are in transmission 
control 407(/). In the preferred embodiment, SMB 507(0 
receives pictures 111 in bursts that contain all or almost all 
of the bits in the picmre, depends on the picture size and 
maximal bit rale specified by the encoder. Such bursts are 
termed herein picture pulses, and the time period represented 
by such a picture pulse is denoted asT^, which is the inverse 
of video frame rate. For example, T^= 1/29.97=33 ms for 
NTSC video coding. As previously stated, packet delivery 
controller 419 provides packets in time slices 211. The 
length of time of one of these slices is denoted herein as T^. 
In a preferred embodiment, T^ is 10 ms. 

SMB 507(0 must of course be large enough to be able to 
accept picture pulses of any size during the time it takes to 
read out the largest expected picture pulse. SMB 507(0 
further must be emptied at a rate that ensures that it cannot 
overflow, since that would result in the loss of bits from bit 
stream 109(0- ^^so should not underflow, since thai would 
result in the insertion of null packets in the bit stream, 
resulting in the waste of a portion of the multiplexed 
medium. Meter 505 monitors the fullness of SMB 507(0 and 
provides information concerning the degree of fullness to 
TRC 413(0- TRC 413(0 Ihen uses this information to vary 
the range of bit rales that it provides to packet delivery 
controller 419 as required to keep SMB 507(0 from over- 
flowing or underflowing. In other embodiments, the degree 
of fuUness from meter 505 can also be fed back to encoder 
107(0 and used there to increase or decrease the encoding 
rate. It should be noted here that feeding back the degree of 
fullness to encoder 107(0 does not create any dependencies 
between statistical multiplexer 401 and a given type of 
encoder 107. Throttle 509, finally, is set by TRC 413 on the 
basis of information 418(0 that it has received from packet 
delivery controUer 419 to indicate the number of packets 113 
that bit stream 109(0 is to provide to medium 207 in time 
slice 211. 

In detenmiaing the range, TRC 413 sets the minimum rate 
for a given time slice 211 to the maximum of the rate 
required to keep SMB 507 from overflowing and the rale 
required to keep VBV model 415(0 &otn underflowing and 
the maximum rate for the time slice to the minimum of the 
rate required to keep SMB 507 from underflowing and the 
rate required to keep VBV model 415(0 from overflowing. 

Continuing with packet delivery controller 419, packet 
delivery controUer 419 allocates the packets 113 that can be 
output during the lime slice 211 T^ to bit streams 109(0 . . .n) 
as required to simultaneously satisfy the ranges of rates and 
priorities provided by TRC 413 for each transmission con- 
troller 407(0 and maximize the number of packets 113 
output during lime slice 211. In the preferred embodiment. 
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controUer 419 has two components, central bit rate control- 
ler 501, which is a processor that analyzes the information 
received from each of the transmission rate controllers 413 
in order to determine how many packets from each bit 
stream 109(0 output in the next time slice 211, and 

switch 511, which takes the number of packets 113 permitted 
by throttle 509(0 for each bit stream 109(0 during the time 
slice 211. Switch 511 is implemented so as to deliver packets 
from each throttle 509(0 such that the packets are evenly 
distributed across time sUce 211. Implementing switch 511 
in this way reduces the burstiness of the stream of packets 
109(0 to decoder 115(0 thereby reduces the amount of 
transport packet buffer needed in decoder 115. Such imple- 
mentations of switch 511 are weU-known in the art. 

An important advantage of multiplexer 401, or indeed of 
any statistical multiplexer built according to the principles of 
Uie invention is that the multiplexer can simultaneously 
muUiplex both constant-rate and variable-rate bit streams 
onto medium 207. The reason for this is that as far as 
statistical multiplexer 401 is concerned, a constant-rate bit 
stream is simply a degenerate case: it is a varying-rate bit 
stream whose rale never varies. Thus, with a constant-rate 
bit stream, TRC 413(0 always returns the same rate infor- 
mation 417(0 10 packet delivery controUer 419. 
Hardware Implementation of a Preferred Embodiment: 
FIGS. 9-11 

A presenUy-preferred embodiment of the invention is 
implemented as a modification of the PowerVu satellite 
up-Unk system manufactured by Scientific-Atlanta, Inc. 
(PowerVu is a trademark of Scientific-Atlanta). FIG. 9 is a 
high-level block diagram of the PowerVu up-link system as 
modified to implement the invenlion. System 901 includes a 
set of encoders 911(0 . . m). Each encoder 911(0 encodes a 
video input 903(0 and an audio input 905(0; video input 
is encoded at a constant or variable bit rate and the audio 
input is encoded at a constant bit rate. Each encoder 911(0 
has an output 913(0 which carries the encoded video and 
audio. In the PowerVu system as modified, the outputs 
913(0 ... /i) go to statistical multiplexer 915, which outputs 
a constant bit-rate stream 917 to a modulator for transmis- 
sion to a communications sateUite. At a high level, operation 
of all of the components of system 901 is supervised and 
conU-oUed by control processor 907, which communicates 
with the other components by means of Ethernet protocol 
909 (Ethernet is a registered trademark of Xerox 
Corporation). In the presently-preferred embodiment, statis- 
tical multiplexer 915 is implemented as a separate chassis 
which need only be coupled to the rest of the PowerVu 
system by encoded data inputs 913(0 . . . /i), Ethernet 
protocol 909, and output 917. 

FIG. 10 shows the preferred embodiment of statistical 
multiplexer 915 in more detail. Multiplexer 915 receives its 
inputs of encoded video and audio from optical fibers. Each 
SWIF receiver 1001(0 receives input from a single optical 
fiber and there are receivers 1001(0 . . . n) corresponding to 
encoders 911(0 ... /»). Each receiver converts the informa- 
tion from photons to digital electronic form and outputs it 
via PCR MOD 1005(0 to channel input block 1009(0- PCR 
MOD 1005(0 corrects the clock information in the encoded 
video and audio to compensate for any delays in the encod- 
ing process. The synchronization information needed to do 
this is provided by MSYNC lock up 1003. 

Channel Input 1009(0 is an implementation of transmis- 
sion controller 407(0 . Channel input 1009(0 employs a 
software implementation of VBV model 415 to dynamicaUy 
determine a current rate at which the input from receiver 
1001(0 tnust be output to multiplexed output stream 917 and 
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provides that rate information to central bit rate controller 
1007, which in turn actually allocates a specific rate to 
channel input block 1009(0- Channel input block 1009(f) 
then outputs bits in its bit stream to bus 1011 at that rate. The 
combined outputs of blocks 1009(0 . . . «) then go via 
multiplexed output 1013, PCR MOD 1016, and SWIF 
transmitter 1017 to output 917. PCR MOD 1016 modifies 
the clock information in the encoded video again to deal 
with the time spent in channel input block 1009(/) and 



Counters 1107 and 1123 are also implemented using gate 
arrays and channel controller 1113 is a digital signal pro- 
cessor. Central bitrate controller 1007 is implemented using 
a microprocessor with a support IC, 
Detailed Description of Algorithms Used to Compute the 
Output Rate for a Bit Stream 109(i) From Statistical Mul- 
tiplexer 401: FIGS. 6, 7, and 12 

As indicated above, the maximum rate R„^ at which a 
transmission controller 407(0 output packets 113 to 



outputs the bit stream to SWIF transmitter 1017, which lo medium 207 is determined by the need to keep SMB buflfer 

■ ' 507(0 from underflowing and bit buffer 119(0 over- 
flowing. The minimum rate R^^ is determined by the need 
to keep SMB buffer 507(0 from overflowing and bit buffer 
119(0 from underflowing. Bit buffer 119(0 will not under- 
15 flow if all packets belonging to the picture currently being 
sent arrive in bit buffer 119(0 before the lime indicated in the 
DTS stamp for the picuire. 

There are thus two maximum rales and two minimum 
rates that need to be taken into account in determining R„^ 



converts the bit stream to a photonic representation and 
outputs it to an optical fiber Communication processor 1015 
provides high level control lo central bilraie controller 1007 
and also serves as the interface lo PCC 907, a control 
console, and a system which broadcasts status informalioo . 
Communications processor 1015 also receives MPEG-2 
service information tables from PCC 907 and provides them 
to service information table insertion 1018, which inserts 
them into the bit streams. 



A presently-preferred embodiment of a single channel 20 and R, 
input block 1009(0 ^ shown in more detail in FIG. 11, The 
main components are packet director 1101, which detects 
audio packets, video packets, and headers and routes them to 
different components of input block 1009(0, storage 1115 
for the headers, storage 1117 for a FIFO (queue) to hold 25 
video packets from the lime they are received in input block 
1009(0 until they are output lo data bus 1011, and a bypass 
FIFO 1119 which holds the constant bit rate audio packets 
while they are in input block 1009(i). Output from FIFO 
1117 is controlled by throttle 1032 under control of throttle 30 
counter 1123, which specifies the number of packets lo be 
output from FIFO 1117 during a given time slot. Output 
from FIFO 1127 is controlled by throttle 1129, which is 

controlled by throttle counter 1123. Throttle counter 1123 is — ■ xraxr 

set bv channel controller 1113 in response to the rate selected 35 What is needed to compute R^^„^ and R^^^ is a VBV model 



R„^i is the maximum rate at which bit buffer 119(0 in 
any MPEG-2 decoder that conforms to the standard 
will not overflow; 

^majc2 is the maximum rale at which SMB 507(0 will not 
underflow; 

Rmmi is the minimum rate at which bit buffer 119(0 will 

not underflow; and 
^mini is the minimum rate at which SMB 507(0 wUl not 

overflow. 

R and R . are determined from the above four maxima 

max mui 

and minima as follows: 



R„,^ is the minimum of R^j and R„ 
R. 



by central bit rate controller 1007. Throttle counter 1127, 
which is for a constant-rate bit stream and does not depend 
on VBV model 415(0, is set directly by central bit rate 
controller 1007. 

Operation of input block 1009(0 is as would be expected. 
Serial bit stream 1001(0 from SWIF receiver 1001(0 is 
modified by PCR 1005(0 is output to packet director 
1101, which delects packets, determines their types, and 
outputs them to the various components of channel input 



block 1009(0- Packet director 1101 funher provides a start 45 more complex. 



415(0 that models the fullness and emptiness of bit buffer 
119(0; what is needed to compute R^,.„2 ^axz is a 
measure of the fullness and emptiness of SMB buffer 507(0- 
The model for the fullness of bh buffer 119(0 is termed 
40 herein VBV fullness and the model for the emptiness of bit 
buffer 11 9(/) is termed herein VBV emptiness. The algo- 
rithms for measuring VBV emptiness and SMB buffer 
emptiness and fullness are simple and will be dealt with first; 
the algorithm for measuring VBV fullness is subslantiaUy 



of picture interrupt 1103 to channel controller 1113 to 
indicate that a new picture is being received in SMB FIFO 
1117. Channel controller 1113 responds to interrupt 1103 by 
using picture size information obtained from picture counter 
1107, header information stored in header storage 1115, and 50 
information about the amount of space left in SMB FIFO 
1117 in the VBV model 415(0 to obtain maximum and 
minimum rates at which data must be output from SMB 
FIFO 1117 lo avoid overflow or underflow in SMB FIFO 
1117 and overflow or underflow in VBV model 415(0* 
Channel controller 1113 outputs these rates via 1121 to 
central bilrate controller 1007, which selecls a rate for the 
next time slice on the basis of the information from channel 
controller 1113, the current output requirements of all of the 
other channel controllers 113, and the total capacity of the 60 to the algorithm below: 
output stream. Central bitrate controUer 1007 returns the 
selected rate lo channel controller 1113, which sets throttle 
counter 1132 accordingly. Throttle counter 1132 then deter- 
mines how many bits are actually output by throttle 1125 
during the next time slice. 65 

As shown in FIG. 11, packet director 1101 is implemented 
by means of gate arrays and a dual port RAM memory. 



In the case of SMB 507(0, the measure of SMB 
emptiness, E^^^, is the amount of free space remaining in 
SMB 507(0- For a given time shoe T^ 211(^?i), it is defined 
as follows: 



£^s>/s=SMB_SIZE-f^wB('")'. 

where F^^^ is the actual SMB fulbess measured by the 
Meter 505. Since there is a maximum size for MPEG-2 
pictures, termed herein VBV_SIZE, the way to prevent 
55 SMB 507(0 frorn overflowing is to guarantee that there is 
always an empO space in SMB 507(0 that is larger than or 
equal to VBV_SIZE. If the free space becomes less than 
that, the minimum rate with regard to SMB 507(0, R„i«2. 
must be increased in the next lime slice T^ (m+1) according 



if C£:svfl<VBV_SIZE){/?„„2(m+l)=(VBV_SIZE-£*,ss(m))/7"c:} 
R^^2 is computed as follows: 

Continuing with the determination of R„i„ j for the next T^ 
from VBV model 415(0, the rate can be found from the 
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information in VBV model 415(z) concerning the pictures 
111 in SMB 507(i). The rule is simply this: the minimum bit 
rate musi be such that the picture currently being output is 
completely output from SMB 507(/) before the time indi- 
cated by its DTS time stamp. One implementation is 5 

^«.«iC'n+l)=pic_rc5iduaLbits iq)}(DTS_V„^-t); 

Here, pic_residuaL„bits is the number of bits of the 
picture 111 remaining in SMB 507(/), q is the index of the 
picture currently being transmitted from SMB 507(/) and 
q+1, q+2, ... are the indexes of the following pictures, 
DTS„V„^ is the time stamp with the most recent time in 
VBV model 415(/), and t is the actual time dctcnnined by the 
synchronization time value in the bit stream. 

The above algorithm guarantees that all bits belonging to 
the picture 111 which is currently being delivered to bit 
buffer 119(0 will have been delivered before the decoding 
time DTS_V^ arrives. This algorithm may leave only one 
coded picture in the decoder's bit buffer for decoding. While 
this picture could be decoded correctly, a high bit rate will 
be necessary to deliver the next picture on time such that all 
the bits belong to the next picture, p+1, will be available for 
decoding at the next decoding lime instance. This require- 
ment will result in a high bitrate requirement for next Tc 
period and will introduce congestion in the delivery media 
at the next Tc period. A better algorithm is one thai guar- 
antees at least two pictures (or more, as long as VBV model 
415(0 ^o^s indicate an overflow) in bit buffer 11 9(/), 
such as the following: 

J H3ic_residuai_bits(i7)/(DTS(9- ] 30 

In this scheme, the minimal bitrate calculation is slightly 
changed by using the second largest value of DTS in bit 
buffer (119(0, DTS (q-1). That is the time stamp for the 
picture 111 preceding the last picture 111 to be sent to bit 
buffer 119(0- This scheme guarantees that the picture p has 
akeady be delivered to decoder 115(0 t=DTS (q-1). Of 
course, it is even belter to set up the minimal bit rate so that 
the number of coded pictures in bit buffer 119(0 is usually 
more than 2. 

Determining VBV Fulbess: FIG. 6 

When there is no need to prevent overflow of SMB 507(0, 
the maximum bitrate of bit stream 109(/) is determined from 
the VBV fullness indicated by VBV model 415(0- 

The greater the VBV fullness indicated by the model, the 
less the maximum bitrale. At the beginning of the operation ^5 
of model 415(0, SMB 507(0 is empty and VBV fullness 
indicates that model 415(0 ^ empty. As soon as bits appear 
in SMB 507(0, central bitrate controller 501 begins output- 
ting them at a predetermined initial rate, for instance, the 
average rale for such variable -rate bit streams. As bits are 
received in SMB 507(0 output to medium 207, the 
picture information in VBV model 415(0 updated each 
time slice. The newly updated information is used to com- 
pute VBV fullness for the next time slice and the VBV 
fullness is used in turn to determine the maximum bit rate 55 
^maxi which bits will be output on bit stream 109(0 for 
the next time period. Th& computation is the following: 

^«..i("'+5>(VBV_SI2E-/^,,>yr, 

where is the VBV fullness measure provided by VBV 60 
model 415(0 ^nd m and m+1 are the current and next lime 
sUcesT^2U. 

In the preferred embodiment, the computation of F^^^ (m) 
is governed by the following considerations: 
The calculation requires a computation of the number of 65 

pictures 111 are currently contained in VBV model 

415(0. 
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The calculation requires a knowledge of how many bits of 
the picture 111 which ts currently being transmitted 
from SMB 507(0 presently remain in SMB 507(0- 
The data items used to compute F^ (m) in the preferred 
embodiment include the following: 

a. \'BV_SIZE, that is, the maximum size of a MPEG-2 
picture. 

b. The absolute maximum bit rale which packet 
delivery controller 419 can provide to bit stream 109(0- 

c. The current time, I, recovered from the clock time 
information of bit stream 109(0- 

d. Data items for each picture presently in SMB 507(0: 
packet_cnt, the number of packets 113 in the picture, 
DTS, the time stamp for the picture, q, the index for 
DTS and packet_cnt for the picture currently leaving 
SMB 507(0, and r, the index for those values for the 
oldest picture for which there is still information in 
model 415(0- 

e. Status data items in VBV model 415(0 lhat are updated 
every T^ 211: pic_cnl__VBV, the number of pictures 
Ul which are presently represented in VBV model 
415(0; pic_residual_bil(q), the number of bits of 
picture 111^ that is currently being transmitted to 
decoder 115(0 that remain in SMB 507(0; DTS_V^^, 
the time stamp with the most recent time stamp value 
that is presently in VBV model 415(0; ^vbv itself- 

As soon as SMB 507(0 begins receiving bit stream 109(0, 
packet delivery controller 419 sets throttle 509(0 *o Ihe 
initial rale provided by central bit rale controller 509. As 
packets are read from SMB 507(0 ^hat rate, transmission 
rate controller 413(0 updates DTS_V„^, pic_cnl_VBY 
F^^, and pic_residual__bits (q) as required by the transmis- 
sion of pictures from SMB 507(0 to decoder 115(0 and by 
the addition of bits to SMB 507(0- The algorithm 601 used 
to do this in a preferred embodiment is shown in FIG. 6. 
Section 603 of algorithm 601 shows how the parameters are 
initialized at the time the first picture arrives in SMB 507(0- 
Execution of loop 604 begins when the first bits of the 
picture arrive in SMB 507(0- As shown at 605, the loop is 
executed once every T^ 211. At the beginning of each 
execution of loop 604, pic_residual_bits is decremented by 
the number of bits that were sent at the rate R(m) previously 
determined for the current T^ 211 by central bitrate control- 
ler 501. 

At 607, F^hv is computed. There are two cases. In the first 
case, shown at 609, the time stamp DTS for the current 
picture r in VBV model 415(0 indicates a lime that is after 
the current time t for bit stream 109(0, so decoding of the 
picture r cannot yet have begun. Consequently, the bits that 
were sent during the last T^ 211 are simply added to the bits 
lhat are already in VBV model 415(0 and F^^,^ is incre- 
mented bad that amount. If the comparison of t and DTS (r) 
indicates that decoder 115(0 has already begun decoding the 
picmre r, the second case, shown at 611, is executed. 
pic_cnt_VBV is decremented to indicate that one less 
picmre is now represented in VBV model 415(0 and F^^^ is 
adjusted by the difference between the number of bits sent 
to decoder 115(0 io the last T^ 211 and the total number of 
bits in the picture that is no longer represented in VBV 
model 415(/). After picUire r is removed from VBV model, 
415(0, Ihe index r is incremented by 1. 

Block of code 613 deals with the updating lhat has to be 
done when a picture q has been completely read from SMB 
507(0- When lhat is the case, pic_residual_bils will have a 
value that is less than or equal to 0. The first updating that 
has to be done is shown at 615. The time stamp DTS for the 
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picture 111 thai was just sent is now the maximurn DTS in Minimal Bitraie Allocation Algorithm, FIG, 12 

Bit buffer 119(/), so DTS_V^^ is updated with DTS (q). A FIG, 12 shows a flowchart 1201 for this algorithm. The 

picture q has also been added to the pictures represented in algorithm allocates a minimal bitrate to eachTRC 413(i) and 

VBV model 415(i), so pic_cni_VBV is incremented returns the number of bits still available to be allocated. The 

accordingly. The second updating is ai 617. The new current 5 allocation is ordered by priorities, beginning with PL=1, as 

picture is the next picture in SMB 507(0, so q is updated shown in block 1201. The remainder of the flowchart 

accordingly. Similarly, pic_residual_bils is set to the num- consists of an inner loop 1215, which is executed for each 

ber of bits in the new current picture. TRC 4l3(i) belonging to a given priority and an outer loop 

Allocating the Total Capacity of Medium 207 Among the 1233 which is executed for each priority. The algorithm 

Channels: FIGS. 7 and 12 10 terminaies when any of three conditions ckccurs: 

FIG, 7 shows a flowchart 701 of the CBC control algo- there is no mort bandwidth to allocate; 

rithm that is used to assign the new bitrate for each VBR ^^^^ ^^^^ ^^j^^^^j ^jit streams 109(0 . . . /i); 

encoder for the next Tc period. The control algonthm is a , r /■ . ... 

loop 713 that executes eacbT .AUhe start of the loop, Rmin ^ aUocations have been made for all of the priorities^ 

and Rmax from each TRC(») are collected. The loul avail- i5 "''"u'"^'^ .-^u'u '^1°^^ ' T u 

able bits per Tc parameter, Be, has already been calculated. 1203, the TRC 413(0 to which bandwidth b currently being 

Be will be only updated when there is a change of channel ^f^') ^^^'^^^ 

bandwidth, Rc, which only happens rarely. Be is calculated ™f 413 i). The bandwidth is rounded to complete ISS-bit 

^ rr packets. In decision block 1205, i! is determmed whether 

there is any bandwidth left. If not, branch 207 is taken, 

Bc-Rc*Tc terminating loop 1215; if there is, loop 1215 continues to 

decision block 1211, where it is determined whether there 

where Tc is in units of seconds. are more bit streams 109(i) having the current priority. If 

Be is divided among the bit stream 109 in accordance with there are, loop 1215 is repealed; otherwise, as indicated by 

the ranges of rales specified by the TRCs (0 . . . /i) and in branch 1213, the program enters a new iteration of outer 

accordance with a set of priorities which indicate which bit " loop 1213. In that loop, decision block 1215 first checks 

streams 109 are more important. The priorities are provided whether there is another priority level to be processed; if 

by the operator of processor 907 and arc set for each bit there is (branch 227), PL is incremented and a new set of 

stream when the multiplexer is initialized for the bit stream. iterations of inner loop 1215 for that priority begins. If there 

In the preferred embodiment, there are three levels of is no additional priority level, loop 1233 terminaies, as seen 

priority, according to the extent to which timely delivery of ' at branch 1229. 

the pictures in the bit stream is required: Looking at the termination conditions in more detail, if 

PL-1: Every picture in the bit stream will be delivered, there is no more bandwidth to be allocated, branch 1207 is 

and each of them will be delivered on time. taken. In decision block 1217, it is determined whether there 

PL=2: Some picture wiU always be delivered on time. For 35 ^re any bit streams 109(0 for which a minimal bandwidth 

example, a picture may be repeated to keep bit buffer °^ust still be allocated. If there are none, branch 1219 is 

115(0 from underflowing. isik&a and the remaining bandwidth is returned at 1235. If 

Di 1 ^»-...«t^^o Tu^ k't there are still bit streams 109(0, ^^e program takes branch 

PL=3: No time guarantees. I ne bit stream could even be ^^^^ , . i^^-y ?• . , t ^ . 

interrupted to give the channel to another bit stream. ^221 and enters the panic proce^ L223 which deals «nth the 

PL 1 and 2 are used for real-lime video programs. PL 3 is 40 '^'i""^'^ ^y the priorities of bit streams 109 0 

J r ♦ ui ^ . .u . J . u- u u 1 n) and then returns the remaining bandwidth at 1235. 

used for preemptible data, that is, data which has no real- J. ^^^^ , ^ ti . - . , ^ 

c„ o..^v, Hn.n nr^ ™«i tir^^ Similarly, branch 1229, taken when all pnonly levels have 

time requirements. Examples 01 such data are non-real lime , . ' ^-*-», 

. . .. J J . ^ . u r 1 been processed, returns the remainme bandwidth at 1235. 

video programs or non-time-dependent data such as E-m ail. . \^ . ^^^^ j- 

PL3 pemiits full use of the available bandwidth in situations Continuing with panic process 1223, if a bu stream 109(0 

where the sum of the video data is less than the total vailable 45 ™^ ^^^^^^ the mmimum rate u requires, one of two 

bandwidth. The total bandwidth available that T, and the ^^"^^ "^^^ depending on the bit stream: 

priority for each bit stream 109(0 is provided by input block SMB 507(0 may overflow, causing loss of data. 

707. The total bandwidth, the priorities, and the maximums bit buffer 119(0 in decoder 115(/) may underflow, causing 

and minimums for the channels are employed in block 705 interruption of the display of pictures, 

to allocate a minimal bit rate to each bit stream 109(0- 50 In the first case, either the input to SMB 507(0 ™ust be 

Details on the algorithm used to do this will be given below. decreased or the output from bit SMB 507(0 tnust be 

Once the minimal bit rates for all bit stream 109(0 . . . n) increased. Generally, the second solution can be employed 

have been allocated, the algorithm subtracts the allocated bit in the short term and the first in the longer term. Beginning 

rates from the total bandwidth to determine whether any with the second solution, the extra bandwidth must be taken 

bandwidth remains (709). If none is left, the allocation is 55 from priority 2 and 3 bit streams, beginning with bit streams 

finished and as shown at 711. 721, and 715, the bandwidth 109(0 with priority 3. These bit streams have no time 

allocated to each TRC 413(0 is assigned to it (721) and loop constraints and can be denied any bandwidth at all for as 

713 is repeated for the next T^. If there are bits left (branch long as is necessary. Bandwidth can also be taken from 

717), the residual bits are assigned to the bit streams 109(0 priority 2 bit streams 109(0 Ibat have space in their SMBs 

that can take more bits (719). The algorithm for doing this 60 507(0 by having them output a repeal of a picture until the 

is also explained in more detail below. Once the residual bits panic condition is over or until their SMB 507(0 threatens 

have been assigned, blocks 701, 715, and loop 213 are to overflow. Of course, what the repeat produces at the 

executed as described above. There remains, of course, the receiver is a still picture. Because the repeat picture is totally 

possibility that there is not enough total bandwidth to redundant with regard to the picture it is repeating, it always 

perform the aUocalion of block 705, This worst-case see- 65 has fewer bits than that picture. 

nario is called Panic mode and will be further discussed Given that the reason for the substitution is to free up 

later. bandwidth, it is desirable to make the repeat picture as small 
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as possible. Thai is achieved by sending a repeal of a coded 
picture that is not used to predict other pictures. B pictures 
fulfill this criterion, as do P pictures that immediately 
precede an I picture in sequences that do not contain B 
pictures. The substitution technique requires that iransrais- 5 
sion controller 113 for a PL 2 bit stream respond to an 
iDdicatioD of a panic from central bitrate controller 1007 by 
reading header information to determine the type and size of 
the picture being output and when it finds the proper kind of 
picture, following it with repeat pictures until the panic is lo 
over. 

Where the problem is underflow of bit buffer 119, if the 
bit stream is a priority 1 bit stream, extra bandwidth must 
again be found and the techniques described above must be 
applied. If bit stream 109(i) is a priority 2 bit stream, the i5 
techniques described for priority 1 bit streams may be 
employed, or if that is not possible, the bandwidth required 
for the bit stream may be reduced by outpulting a minimal- 
sized repeat picture as described above until the panic 
condition is over or until overflow of SMB 507(/) threatens. 20 

Where the problem is the threatened overflow of one or 
more SMB buffers 507, it may also be addressed by decreas- 
ing the bit rale at which the encoders 107 produce data. If the 
encoders 107 are co-located with statistical multiplexer 401, 
feedback from multiplexer 401 to the encoders may be used 25 
to do this. With this kind of feedback, there is no require- 
ment that multiplexer 401 understand the inner workings of 
encoders 107. All that the signal to a given encoder 107(/) 
need indicate is that the encoder must reduce its output rate 
by some amount. Which encoders receive the signal can be 30 
determined in many fashions by multiplexer 401. One 
approach is to reduce the bit rale, (and therefore the image 
quality) in channels on the basis of their priority levels; 
another is to reduce the bit rale in ail channels equally. 
Typically, taking bandwidth from other bit streams would be 35 
a short-tenn solution that would be employed until the 
encoding rate could be changed. In the preferred hardware 
embodiment, short-term panic management is done in cen- 
tral bitrale controller 1007, while long-term panic manage- 
ment is done in control processor 907. 40 
Algorithm for Allocating Residual Bits 

When each of the bit streams 109(/) has received its 
minimum bilraie and there is still bandwidth remaining in 
medium 207, this residual bandwidth B, is allocated among 
the bit streams in the preferred embodiment by allocating 45 
each bit stream 109(i) an additional bit rate AR(/) which is 
proportional to the difference between the maximum and 
minimum bit rates computed by TRC 413(/) for the bit 
stream. AR(/) is calculated in the preferred embodiment as 
follows: 
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In a preferred embodimeni, all of ihe bit rales involved in the 
above computation are rounded to an integer number of 
packets per second. 

CONCLUSION 

The foregoing Detailed Description has disclosed to those 
skilled in the arts to which the invention pertains how to 
make and use apparatus for outpulting a variable-rale bit 
stream which uses a model of a receiver of the bit stream and 
information from the bit stream lo determine a rale at which 
the variable-rate bit stream must be output to avoid overflow 
or underflow in the receiver. The Detailed Description has 
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further disclosed a multiplexer which employs the apparams 
10 multiplex a set of variable- rale bit streams onto a 
medium, has disclosed how such apparatus and multiplexers 
may be used with bit streams of video images that are 
encoded according to the MPEG-2 standard, and has given 
algorithms for the use of models of MPEG-2 receivers to 
compute rate requirements. 

The Detailed Description has disclosed the best mode 
presently known lo the inventors of implementing their 
apparatus and multiplexers which employ the apparatus; it 
will, however, be immediately apparent lo those skilled in 
the arts lo which the invention pertains that the invention 
may be employed with variable-rate bit streams other than 
those which are encoded according lo the MPEG-2 standard 
and that many other implementations can be made which 
will incorporate the principles of the invention but employ 
different specific implementation techniques. 

For these reasons, the Detailed Description is to be 
regarded as being in all respects exemplary and not 
restrictive, and the breadth of the invention disclosed herein 
is lo be determined not from the Detailed Description, but 
rather from the claims as interpreted with the full breadth 
permitted by the patent laws. 

What is claimed is; 

1. An apparatus for controlling a rate at which a bit stream 
encoded at a variable rate is output, the apparatus compris- 
ing: 

a queue for receiving and storing the bit stream; 

a processor for applying a model of a receiver for the bil 
stream lo information read from the bil stream lo 
determine an output rate for outpulting the bit stream; 

a bit stream analyzer for analyzing at least timing infor- 
mation and picture size information in said bit stream; 
and 

output apparatus responsive lo the output rale for output- 
ting the bil stream from the queue. 

2. The apparatus set forth in claim 1 wherein: 

the processor determines the output rate such that the 
receiver of the model will neither overflow nor under- 
flow. 

3. The apparatus sel forth in claim 2 wherein: 

the processor is further responsive to a state of the queue 
and further determines the output rate such that the 
queue will not overflow. 

4. The apparatus sel forth in claim 3 wherein: 

the processor further determines the output rate such thai 
the queue will not underflow. 

5. The apparatus sel forth in any of claims 1 through 4 
wherein: 

the bil stream is a sequence of components, the compo- 
nents having varying lengths and each component 
including liming information indicating when the 
receiver must process the componeni and 

the model includes 

a current sel of components which is the set of the 
components in the queue together with the set of the 
components that have been sent to but not yel 
processed by the receiver, 

the timing information for the components in the cur- 
rent set, 

the sizes of the componcnls in the current sel, and 
a size of a bit buffer in the receiver of the model. 

6. The apparatus sel forth in claim 5 wherein: 

the components are digitally -encoded video images. 
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7. The apparatus set forth in claim 6 wherein: 

the video images are encoded according to the MPEG-2 
standard. 

8. The apparatus set forth in claim 2 wherein: 

the bit stream is a sequence of components, the compo- ^ 
nents having varying lengths and each component 
including timing information indicating when the 
receiver must process the component; 
the model includes jq 
a current set of components which is the set of the 

components in the queue together with the set of the 

components that have been sent to but not yet 

processed by the receiver, 
the timing information for the components in the cur- 

real set, 

the sizes of the components in the current set, and 
a size of a bit buffer in the receiver of the model; and 
the processor determines the output rale for a given period 
of time by determining a minimum rate such thai each 20 
component is output from the queue before the receiver 
must process the component and a maximum rate such 
that the total size of the components in the set of the 
components that have been sent to but not yet pro- 
cessed by the receiver does not exceed the size of the 25 
bit buffer in the receiver of the model, 

9. The apparatus set forth in claim 8 wherein: 

the processor increases Ihe minimum rate for the given 
time period according to the number of bits that remain 
in the queue from a current component that is currently 30 
being output from the queue and the amount of time 
remaining until the current component must have been 
output from the queue and 

the processor decreases the maximum rate for the given 
time period as the total size of the components in the set 
of components that have been sent to the receiver and 
not yet processed approaches the size of the bit buffer 
in the receiver of the model. 

10. The apparatus set forth in claim 1, further including a 
bandwidth portion controller coupled to said output appa- 
ratus. 

The apparatus set forth in claim 10, wherein said 
bandwidth portion controller dynamically determines a vari- 
able portion of said bit stream to be output lo a medium. 

12. The apparatus set forth is claim 10, wherein said 
bandwidth portion controller is coupled to a plurality of 
output apparatus. 

13. A method of dynamically determining an output rate 
at which a bit stream encoded at a variable rate is output to 
a receiver, the method comprising the steps of: 

receiving and storing the bit stream in a queue; 
analyzing said bit stream for at least timing information 

and picture size information; 
in a processor, applying a model of the receiver to 55 

information read from the bit stream to delemqine the 

output rate; and 
outputtingthe bit stream from the queue at the output rale. 

14. The method set forth in claim 13 wherein: 

the output rale is determined such that the receiver of the ^0 
model will neither overflow nor underflow. 

15. The method set forth in claim 14 wherein the method 
further comprises the step of: 

determining a state of the queue; and 
the step of determining the output rate further determines 
the ouiput rale such that the queue will not overflow. 
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16. The method set forth in claun 15 wherein: 

the step of determining the output rate fiu"ther determines 
the output rate such that the queue will not underflow. 

17. The method set forth in any of claims 13 through 16 
wherein 

the bit stream is a sequence of components, the compo- 
nents having varying lengths and each component 
including timing information indicating when the 
receiver must process the component, 
the model includes 

a current set of components which is the set of the 
components in the queue together with Uie set of the 
components that have been sent to but not yet 
processed by the receiver, 
the timing information for the components in the cur- 
rent set, 

the sizes of the components in the current set, and 
a size of a bit buffer in the receiver of the model, and 
the step of determining the output rate is done for a given 
period of time and comprises the steps of 
determining a minimum rate such that the component is 
output from the queue before the receiver must 
process the component and 
determining a maximum rate such that the total size of 
the components in the set of the components that 
have been sent to but not yet processed by the 
receiver does not exceed the size of the bit buffer in 
the receiver of the model. 

18. The method set forth in claim 17 wherein: 

the step of determining the minimum rate increases the 
minimum rate for the given time period according to 
the number of bits that remain in the queue from a 
current component that is currently being output from 
the queue and the amount of lime remaining until the 
current component must have been output from the 
queue and 

the step of determining the maximum rate decreases the 
maximum rate for the given time period as the total size 
of the components in the set of components thai have 
been sent to the receiver and not yet processed 
approaches the size of the bit buffer in the receiver of 
the model. 

19. The method set forth in claim 17 wherein: 

the components are digitally-encoded video images. 

20. The method set forth in claim 19 wherein: 

the video images are encoded according to the MPEG-2 
standard. 

21. An apparatus for controlling a rate at which an 
encoder encodes a bit stream, the apparatus comprising: 

a queue for receiving and storing the bit stream from the 
encoder; 

a bit stream analyzer for analyzing at least timing infor- 
mation and picture size information in said bit stream; 
and 

a processor coupled to the bit stream and to the encoder 
for applying a model of a receiver for the bit stream to 
the information read from the bit stream to determine 
the rate at which the encoder encodes ihe bit stream. 

22. The apparatus as set forth in claim 21, further includ- 
ing a bandwidth portion controller. 

23. The apparatus as set forth in claim 22, wherein said 
bandwidth portion controller dynamically determines a vari- 
able portion of said bit stream to be output to a medium. 

« * « * * 
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